Removing airbag modules from automotive scrap

ABSTRACT

A system classifies materials utilizing a vision system that implements an artificial intelligence system in order to identify or classify and then remove automotive airbag modules from a scrap stream, which may have been produced from a shredding of end-of-life vehicles. The sorting process may be designed so that live airbag modules are not activated, which may cause damage to equipment or persons.

RELATED PATENTS AND PATENT APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationSer. No. 63/229,724. This application is a continuation-in-partapplication of U.S. patent application Ser. No. 17/752,669, which is acontinuation-in-part application of U.S. patent application Ser. No.17/667,397, which is a continuation-in-part application of U.S. patentapplication Ser. No. 17/495,291, which is a continuation-in-partapplication of U.S. patent application Ser. No. 17/491,415 (issued asU.S. Pat. No. 11,278,937), which is a continuation-in-part applicationof U.S. patent application Ser. No. 17/380,928, which is acontinuation-in-part application of U.S. patent application Ser. No.17/227,245, which is a continuation-in-part application of U.S. patentapplication Ser. No. 16/939,011, which is a continuation application ofU.S. patent application Ser. No. 16/375,675 (issued as U.S. Pat. No.10,722,922), which is a continuation-in-part application of U.S. patentapplication Ser. No. 15/963,755 (issued as U.S. Pat. No. 10,710,119),which is a continuation-in-part application of U.S. patent applicationSer. No. 15/213,129 (issued as U.S. Pat. No. 10,207,296), which claimspriority to U.S. Provisional Patent Application Ser. No. 62/193,332, allof which are hereby incorporated by reference herein. U.S. patentapplication Ser. No. 17/491,415 (issued as U.S. Pat. No. 11,278,937) isa continuation-in-part application of U.S. patent application Ser. No.16/852,514 (issued as U.S. Pat. No. 11,260,426), which is a divisionalapplication of U.S. patent application Ser. No. 16/358,374 (issued asU.S. Pat. No. 10,625,304), which is a continuation-in-part applicationof U.S. patent application Ser. No. 15/963,755 (issued as U.S. Pat. No.10,710,119), which claims priority to U.S. Provisional PatentApplication Ser. No. 62/490,219, all of which are hereby incorporated byreference herein.

GOVERNMENT LICENSE RIGHTS

This disclosure was made with U.S. government support under Grant No.DE-AR0000422 awarded by the U.S. Department of Energy. The U.S.government may have certain rights in this disclosure.

TECHNICAL FIELD

This invention relates to recycling of automotive scrap, and moreparticularly to the removal of airbag modules from automotive scrap.

BACKGROUND

This section is intended to introduce various aspects of the art, whichmay be associated with exemplary embodiments of the present disclosure.This discussion is believed to assist in providing a framework tofacilitate a better understanding of particular aspects of the presentdisclosure. Accordingly, it should be understood that this sectionshould be read in this light, and not necessarily as admissions of priorart.

Recycling is the process of collecting and processing materials thatwould otherwise be thrown away as trash, and turning them into newproducts. Recycling has benefits for communities and for theenvironment, since it reduces the amount of waste sent to landfills andincinerators, conserves natural resources, increases economic securityby tapping a domestic source of materials, prevents pollution byreducing the need to collect new raw materials, and saves energy.

Scrap metals are often shredded, and thus require sorting to facilitatereuse of the metals. By sorting the scrap metals, metal is reused thatmay otherwise go to a landfill. Additionally, use of sorted scrap metalleads to reduced pollution and emissions in comparison to refiningvirgin feedstock from ore. Scrap metals may be used in place of virginfeedstock by manufacturers if the quality of the sorted metal meetscertain standards. The scrap metals may include types of ferrous andnonferrous metals, heavy metals, high value metals such as nickel ortitanium, cast or wrought metals, and other various alloys.

An estimated fifteen million vehicles are shredded in the U.S. each year(often referred to as end-of-life vehicles). Each vehicle may haveseveral (e.g., 6-15) airbag modules; that is more than ninety millionairbag modules that may enter the automotive recycling streams eachyear. An air bag module typically has three main parts enclosed withinsome sort of container or canister: the air bag, the inflator, and thepropellant.

A resulting problem associated with all of these airbag modules is thatthey contain sodium azide, used for inflation, which is toxic.Additionally, as the airbag modules pass through the vehicle shredder,not all of them inflate/explode. Consequently, those airbag modules mayinflate/explode in different locations with different consequences: onconveyor systems, damaging the conveyor belt; while being handled bypeople, with possible severe injuries and/or loss of limbs; and afterbeing sold from a recycling facility to a customer, damaging customerequipment.

There are technical challenges to overcome for ensuring the satisfactoryremoval of such airbag modules from the vehicle scrap: airbag modulescan be small (e.g., airbag modules often have a form factor of one inchdiameter cylinders that are one inch in height); airbag modules in amixed scrap metal stream after the shredding process appear similar toother pieces of scrap metal; the airbag modules after shredding aredifficult to identify when mixed with the other scrap pieces; the airbagmodules can be partially occluded while being transported on a conveyorbelt when mixed in with the other scrap pieces; and airbag modules comein different shapes, sizes, and colors.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a schematic of a material handling system configuredin accordance with embodiments of the present disclosure.

FIGS. 2A-2B illustrate exemplary representations of control sets ofairbag modules used during a training stage.

FIG. 2C illustrates an exemplary representation of a control set ofairbag modules used during a training stage in which the algorithm hasidentified and classified the airbag modules.

FIG. 3 illustrates a flowchart diagram configured in accordance withembodiments of the present disclosure.

FIG. 4 illustrates a flowchart diagram configured in accordance withembodiments of the present disclosure.

FIG. 5 illustrates a block diagram of a data processing systemconfigured in accordance with embodiments of the present disclosure.

FIGS. 6A-6B illustrate exemplary representations of heterogeneousmixtures of material pieces that include airbag modules.

FIG. 7 illustrates an exemplary representation of a heterogeneousmixture of material pieces that include airbag modules in which anartificial intelligence algorithm has identified and/or classified theairbag modules.

DETAILED DESCRIPTION

Various detailed embodiments of the present disclosure are disclosedherein. However, it is to be understood that the disclosed embodimentsare merely exemplary of the disclosure, which may be embodied in variousand alternative forms. The figures are not necessarily to scale; somefeatures may be exaggerated or minimized to show details of particularcomponents. Therefore, specific structural and functional detailsdisclosed herein are not to be interpreted as limiting, but merely as arepresentative basis for teaching one skilled in the art to employvarious embodiments of the present disclosure.

Embodiments of the present disclosure utilize artificial intelligencetechniques for identification/classification of airbag modules in ascrap stream. In accordance with certain embodiments of the presentdisclosure, the material pieces can be separated on a conveyor belt withspaces between pieces using any standard computer vision methods. Inaccordance with certain embodiments of the present disclosure, a regionproposal neural network may be utilized for detection of the airbagmodules, and/or a deep neural network may be utilized for classificationof the airbag modules. In accordance with certain embodiments of thepresent disclosure, these two neural networks for detection andclassification may be combined. Embodiments of the present disclosuremay use semantic segmentation or object detection/localization.Alternatively, instance segmentation or panoptic segmentation may beutilized. Embodiments of the present disclosure may use pixel-level,neighborhood, regional, and/or whole-image classification.

As used herein, “materials” may include any item or object, includingbut not limited to, metals (ferrous and nonferrous), metal alloys,pieces of metal embedded in another different material, plastics(including, but not limited to any of the plastics disclosed herein,known in the industry, or newly created in the future), rubber, foam,glass (including, but not limited to borosilicate or soda lime glass,and various colored glass), ceramics, paper, cardboard, Teflon, PE,bundled wires, insulation covered wires, rare earth elements, leaves,wood, plants, parts of plants, textiles, bio-waste, packaging,electronic waste, batteries and accumulators, automotive scrap piecesfrom shredded vehicles, mining, construction, and demolition waste, cropwastes, forest residues, purpose-grown grasses, woody energy crops,microalgae, urban food waste, food waste, hazardous chemical andbiomedical wastes, construction debris, farm wastes, biogenic items,non-biogenic items, objects with a specific carbon content, any otherobjects that may be found within municipal solid waste, and any otherobjects, items, or materials disclosed herein, including further typesor classes of any of the foregoing that can be distinguished from eachother, including but not limited to, by one or more sensor systems,including but not limited to, any of the sensor technologies disclosedherein.

In a more general sense, a “material” may include any item or objectcomposed of a chemical element, a compound or mixture of one or morechemical elements, or a compound or mixture of a compound or mixture ofchemical elements, wherein the complexity of a compound or mixture mayrange from being simple to complex (all of which may also be referred toherein as a material having a particular “chemical composition”).“Chemical element” means a chemical element of the periodic table ofchemical elements, including chemical elements that may be discoveredafter the filing date of this application. Within this disclosure, theterms “scrap,” “scrap pieces,” “materials,” and “material pieces” may beused interchangeably. As used herein, a material piece or scrap piecereferred to as having a metal alloy composition is a metal alloy havinga particular chemical composition that distinguishes it from other metalalloys.

As used herein, the term “predetermined” refers to something that hasbeen established or decided in advance, including, but not limited to,by a user or operator of a sorting system as disclosed herein.

As used herein, “spectral imaging” is imaging that uses multiple bandsacross the electromagnetic spectrum. While a typical camera captureslight across three wavelength bands in the visible spectrum, red, green,and blue (“RGB”), spectral imaging may encompass a wide variety oftechniques that include and go beyond RGB. For example, spectral imagingmay use the infrared, visible, ultraviolet, and/or x-ray spectrums, orsome combination of the above. Spectral data, or spectral image data, isa digital data representation of a spectral image. Spectral imaging mayinclude the simultaneous acquisition of spectral data in visible andnon-visible bands, illumination from outside the visible range, or theuse of optical filters to capture a specific spectral range. It is alsopossible to capture hundreds of wavelength bands for each pixel in aspectral image.

As used herein, the term “image data packet” refers to a packet ofdigital data pertaining to a captured spectral image of an individualmaterial piece.

As used herein, the terms “identify” and “classify,” the terms“identification” and “classification,” and any derivatives of theforegoing, may be utilized interchangeably. As used herein, to“classify” a piece of material is to determine (i.e., identify) a typeor class of materials to which the piece of material belongs. Forexample, in accordance with certain embodiments of the presentdisclosure, a sensor system (as further described herein) may beconfigured to collect and analyze any type of information forclassifying materials, which classifications can be utilized within asorting system to selectively sort material pieces as a function of aset of one or more physical and/or chemical characteristics (e.g., whichmay be user-defined), including but not limited to, color, texture, hue,shape, brightness, weight, density, chemical composition, size,uniformity, manufacturing type, chemical signature, predeterminedfraction, radioactive signature, transmissivity to light, sound, orother signals, and reaction to stimuli such as various fields, includingemitted and/or reflected electromagnetic radiation (“EM”) of thematerial pieces.

The types or classes (i.e., classification) of materials may beuser-definable and not limited to any known classification of materials.The granularity of the types or classes may range from very coarse tovery fine. For example, the types or classes may include plastics,ceramics, glasses, metals, and other materials, where the granularity ofsuch types or classes is relatively coarse; different metals and metalalloys such as, for example, zinc, copper, brass, chrome plate, andaluminum, where the granularity of such types or classes is finer; orbetween specific types of plastic, where the granularity of such typesor classes is relatively fine. Thus, the types or classes may beconfigured to distinguish between materials of significantly differentchemical compositions such as, for example, plastics and metal alloys,or to distinguish between materials of almost identical chemicalcompositions such as, for example, different types of metal alloys. Itshould be appreciated that the methods and systems discussed herein maybe applied to accurately identify/classify pieces of material for whichthe chemical composition is completely unknown before being classified.

As referred to herein, a “conveyor system” may be any known piece ofmechanical handling equipment that moves materials from one location toanother, including, but not limited to, an aero-mechanical conveyor,automotive conveyor, belt conveyor, belt-driven live roller conveyor,bucket conveyor, chain conveyor, chain-driven live roller conveyor, dragconveyor, dust-proof conveyor, electric track vehicle system, flexibleconveyor, gravity conveyor, gravity skatewheel conveyor, lineshaftroller conveyor, motorized-drive roller conveyor, overhead I-beamconveyor, overland conveyor, pharmaceutical conveyor, plastic beltconveyor, pneumatic conveyor, screw or auger conveyor, spiral conveyor,tubular gallery conveyor, vertical conveyor, free-fall conveyor,vibrating conveyor, wire mesh conveyor, and robotic arm manipulators.

The systems and methods described herein according to certainembodiments of the present disclosure receive a heterogeneous mixture ofa plurality of material pieces, wherein at least one material piecewithin this heterogeneous mixture includes a composition of elementsdifferent from one or more other material pieces and/or at least onematerial piece within this heterogeneous mixture is physicallydistinguishable from other material pieces, and/or at least one materialpiece within this heterogeneous mixture is of a class or type ofmaterial different from the other material pieces within the mixture,and the systems and methods are configured toidentify/classify/distinguish/sort this one material piece into a groupseparate from such other material pieces. Embodiments of the presentdisclosure may be utilized to sort any types or classes of materials asdefined herein. By way of contrast, a homogeneous set or group ofmaterials all fall within an identifiable class or type of material (or,even a specified plurality of identifiable classes or types ofmaterials), such as live airbag modules.

Embodiments of the present disclosure may be described herein as sortingmaterial pieces into such separate groups by physically depositing(e.g., diverting or ejecting) the material pieces into separatereceptacles or bins as a function of user-defined groupings (e.g., typesor classifications of materials). As an example, within certainembodiments of the present disclosure, material pieces may be sortedinto separate receptacles in order to separate material piecesclassified as belonging to a certain class or type of material (e.g.,live airbag modules) that are distinguishable from other material pieces(for example, which are classified as belonging to a different class ortype of material).

It should be noted that the materials to be sorted may have irregularsizes and shapes. For example, such materials may have been previouslyrun through some sort of shredding mechanism that chops up the materialsinto such irregularly shaped and sized pieces (producing scrap pieces),which may then be fed or diverted onto a conveyor system. In accordancewith embodiments of the present disclosure, the material pieces includeautomotive scrap pieces of vehicles, which have been passed through somesort of shredding mechanism, wherein the automotive scrap pieces includeairbag modules that have not been activated (i.e., inflated orexploded), which are also referred to herein as “live airbag modules.”

FIG. 1 illustrates an example of a system 100 configured in accordancewith various embodiments of the present disclosure. A conveyor system103 may be implemented to convey individual material pieces 101 throughthe system 100 so that each of the individual material pieces 101 can betracked, classified, distinguished, and sorted into predetermineddesired groups. Such a conveyor system 103 may be implemented with oneor more conveyor belts on which the material pieces 101 travel,typically at a predetermined constant speed. However, certainembodiments of the present disclosure may be implemented with othertypes of conveyor systems, including a system in which the materialpieces free fall past the various components of the system 100 (or anyother type of vertical sorter), or a vibrating conveyor system.Hereinafter, wherein applicable, the conveyor system 103 may also bereferred to as the conveyor belt 103. In one or more embodiments, someor all of the acts or functions of conveying, capturing, stimulating,detecting, classifying, distinguishing, and sorting may be performedautomatically, i.e., without human intervention. For example, in thesystem 100, one or more cameras, one or more sources of stimuli, one ormore emissions detectors, a classification module, a sortingdevice/apparatus, and/or other system components may be configured toperform these and other operations automatically.

Furthermore, though FIG. 1 illustrates a single stream of materialpieces 101 on a conveyor system 103, embodiments of the presentdisclosure may be implemented in which a plurality of such streams ofmaterial pieces are passing by the various components of the system 100in parallel with each other. For example, as further described in U.S.Pat. No. 10,207,296, the material pieces may be distributed into two ormore parallel singulated streams travelling on a single conveyor belt,or a set of parallel conveyor belts. As such, certain embodiments of thepresent disclosure are capable of simultaneously tracking, classifying,and sorting a plurality of such parallel travelling streams of materialpieces. In accordance with certain embodiments of the presentdisclosure, incorporation or use of a singulator is not required.Instead, the conveyor system (e.g., the conveyor belt 103) may simplyconvey a collection of material pieces, which may have been depositedonto the conveyor system 103 in a random manner.

In accordance with certain embodiments of the present disclosure, somesort of suitable feeder mechanism (e.g., another conveyor system orhopper 102) may be utilized to feed the material pieces 101 onto theconveyor system 103, whereby the conveyor system 103 conveys thematerial pieces 101 past various components within the system 100. Afterthe material pieces 101 are received by the conveyor system 103, anoptional tumbler/vibrator/singulator 106 may be utilized to separate theindividual material pieces from a mass of material pieces. Withincertain embodiments of the present disclosure, the conveyor system 103is operated to travel at a predetermined speed by a conveyor systemmotor 104. This predetermined speed may be programmable and/oradjustable by the operator in any well-known manner. Monitoring of thepredetermined speed of the conveyor system 103 may alternatively beperformed with a position detector 105. Within certain embodiments ofthe present disclosure, control of the conveyor system motor 104 and/orthe position detector 105 may be performed by an automation controlsystem 108. Such an automation control system 108 may be operated underthe control of a computer system 107, and/or the functions forperforming the automation control may be implemented in software withinthe computer system 107.

The conveyor system 103 may be a conventional endless belt conveyoremploying a conventional drive motor 104 suitable to move the beltconveyor at the predetermined speeds. The position detector 105, whichmay be a conventional encoder, may be operatively coupled to theconveyor system 103 and the automation control system 108 to provideinformation corresponding to the movement (e.g., speed) of the conveyorbelt. Thus, through utilization of the controls to the conveyor systemdrive motor 104 and/or the automation control system 108 (andalternatively including the position detector 105), as each of thematerial pieces 101 travelling on the conveyor system 103 areidentified, they can be tracked by location and time (relative to thevarious components of the system 100) so that various components of thesystem 100 can be activated/deactivated as each material piece 101passes within their vicinity. As a result, the automation control system108 is able to track the location of each of the material pieces 101while they travel along the conveyor system 103.

Referring again to FIG. 1, certain embodiments of the present disclosuremay utilize a vision, or optical recognition, system 110 and/or amaterial piece tracking device 111 as a means to track each of thematerial pieces 101 as they travel on the conveyor system 103. Thevision system 110 may utilize one or more still or live action cameras109 to note the position (i.e., location and timing) of each of thematerial pieces 101 on the moving conveyor system 103. The vision system110 may be further, or alternatively, configured to perform certaintypes of identification (e.g., classification) of all or a portion ofthe material pieces 101, as will be further described herein. Forexample, such a vision system 110 may be utilized to capture or acquireinformation about each of the material pieces 101. For example, thevision system 110 may be configured (e.g., with an artificialintelligence (“AI”) system) to capture or collect any type ofinformation from the material pieces that can be utilized within thesystem 100 to classify/distinguish and selectively sort the materialpieces 101 as a function of a set of one or more characteristics (e.g.,physical and/or chemical and/or radioactive, etc.) as described herein.In accordance with certain embodiments of the present disclosure, thevision system 110 may be configured to capture visual images of each ofthe material pieces 101 (including one-dimensional, two-dimensional,three-dimensional, or holographic imaging), for example, by using anoptical sensor as utilized in typical digital cameras and videoequipment. Such visual images captured by the optical sensor are thenstored in a memory device as spectral image data (e.g., formatted asimage data packets). In accordance with certain embodiments of thepresent disclosure, such spectral image data may represent imagescaptured within optical wavelengths of light (i.e., the wavelengths oflight that are observable by the typical human eye). However,alternative embodiments of the present disclosure may utilize sensorsystems that are configured to capture an image of a material made up ofwavelengths of light outside of the visual wavelengths of the human eye.

In accordance with alternative embodiments of the present disclosure,the system 100 may be implemented with one or more sensor systems 120,which may be utilized solely or in combination with the vision system110 to classify/identify/distinguish material pieces 101. A sensorsystem 120 may be configured with any type of sensor technology,including sensors utilizing irradiated or reflected electromagneticradiation (e.g., utilizing infrared (“IR”), Fourier Transform IR(“FTIR”), Forward-looking Infrared (“FLIR”), Very Near Infrared(“VNIR”), Near Infrared (“NIR”), Short Wavelength Infrared (“SWIR”),Long Wavelength Infrared (“LWIR”), Medium Wavelength Infrared (“MWIR” or“MIR”), X-Ray Transmission (“XRT”), Gamma Ray, Ultraviolet (“UV”), X-RayFluorescence (“XRF”), Laser Induced Breakdown Spectroscopy (“LIBS”),Raman Spectroscopy, Anti-stokes Raman Spectroscopy, Gamma Spectroscopy,Hyperspectral Spectroscopy (e.g., any range beyond visible wavelengths),Acoustic Spectroscopy, NMR Spectroscopy, Microwave Spectroscopy,Terahertz Spectroscopy, including one-dimensional, two-dimensional, orthree-dimensional imaging with any of the foregoing), or by any othertype of sensor technology, including but not limited to, chemical orradioactive. Implementation of an XRF system (e.g., for use as a sensorsystem 120 herein) is further described in U.S. Pat. No. 10,207,296.Note that, in certain contexts of the description herein, reference to asensor system thus may refer to a vision system. Nevertheless, any ofthe vision and sensor systems disclosed herein may be configured tocollect or capture information (e.g., characteristics) particularlyassociated with each of the material pieces, whereby that capturedinformation may then be utilized to identify/classify/distinguishcertain ones of the materials pieces.

In accordance with certain embodiments of the present disclosure, morethan one optical camera and/or sensor system may be used, including atdifferent angles, to help identify live airbag modules partiallyoccluded, or even substantially or totally occluded, by other materialson the conveyor system. In accordance with certain embodiments of thepresent disclosure, multiple cameras and/or sensor systems may be usedto create 3D information to generate more usable information thanpossible with 2D data. The 2D or 3D data can be used with AI system togather the data.

In accordance with certain embodiments of the present disclosure, aLidar system (“light detection and ranging” or “laser imaging,detection, and ranging”) can be used instead of a camera and/or a sensorsystem. In accordance with certain embodiments of the presentdisclosure, a scanning laser can be used to gather 3D data of the scrapstream. The laser-based 3D data may then be used with a neural networkto identify the live airbag modules.

It should be noted that though FIG. 1 is illustrated with a combinationof a vision system 110 and one or more sensor systems 120, embodimentsof the present disclosure may be implemented with any combination ofsensor systems utilizing any of the sensor technologies disclosedherein, or any other sensor technologies currently available ordeveloped in the future. Though FIG. 1 is illustrated as including oneor more sensor systems 120, implementation of such sensor system(s) isoptional within certain embodiments of the present disclosure. Withincertain embodiments of the present disclosure, a combination of both thevision system 110 and one or more sensor systems 120 may be used toclassify the material pieces 101. Within certain embodiments of thepresent disclosure, any combination of one or more of the differentsensor technologies disclosed herein may be used to classify thematerial pieces 101 without utilization of a vision system 110.Furthermore, embodiments of the present disclosure may include anycombinations of one or more sensor systems and/or vision systems inwhich the outputs of such sensor/vision systems are processed within anAI system (as further disclosed herein) in order to classify/identifymaterials from a heterogeneous mixture of materials, which can then besorted from each other.

Within certain embodiments of the present disclosure, the material piecetracking device 111 and accompanying control system 112 may be utilizedand configured to measure the sizes and/or shapes of each of thematerial pieces 101 as they pass within proximity of the material piecetracking device 111, along with the position (i.e., location and timing)of each of the material pieces 101 on the moving conveyor system 103. Anexemplary operation of such a material piece tracking device 111 andcontrol system 112 is further described in U.S. Pat. No. 10,207,296.Alternatively, as previously disclosed, the vision system 110 may beutilized to track the position (i.e., location and timing) of each ofthe material pieces 101 as they are transported by the conveyor system103. As such, certain embodiments of the present disclosure may beimplemented without a material piece tracking device (e.g., the materialpiece tracking device 111) to track the material pieces.

Such a distance measuring device 111 may be implemented with awell-known visible light (e.g., laser light) system, which continuouslymeasures a distance the light travels before being reflected back into adetector of the laser light system. As such, as each of the materialpieces 101 passes within proximity of the device 111, it outputs asignal to the control system 112 indicating such distance measurements.Therefore, such a signal may substantially represent an intermittentseries of pulses whereby the baseline of the signal is produced as aresult of a measurement of the distance between the distance measuringdevice 111 and the conveyor belt 103 during those moments when amaterial piece 101 is not in the proximity of the device 111, while eachpulse provides a measurement of the distance between the distancemeasuring device 111 and a material piece 101 passing by on the conveyorbelt 103.

Within certain embodiments of the present disclosure that implement oneor more sensor systems 120, the sensor system(s) 120 may be configuredto assist the vision system 110 to identify the chemical composition,relative chemical compositions, and/or manufacturing types, of each ofthe material pieces 101 as they pass within proximity of the sensorsystem(s) 120. The sensor system(s) 120 may include an energy emittingsource 121, which may be powered by a power supply 122, for example, inorder to stimulate a response from each of the material pieces 101.

Within certain embodiments of the present disclosure, as each materialpiece 101 passes within proximity to the emitting source 121, the sensorsystem 120 may emit an appropriate sensing signal towards the materialpiece 101. One or more detectors 124 may be positioned and configured tosense/detect one or more characteristics from the material piece 101 ina form appropriate for the type of utilized sensor technology. The oneor more detectors 124 and the associated detector electronics 125capture these received sensed characteristics to perform signalprocessing thereon and produce digitized information representing thesensed characteristics (e.g., spectral data), which is then analyzed inaccordance with certain embodiments of the present disclosure, which maybe used to classify each of the material pieces 101. Thisclassification, which may be performed within the computer system 107,may then be utilized by the automation control system 108 to activateone of the N (N≥1) sorting devices 126 . . . 129 of a sorting apparatusfor sorting (e.g., removing/diverting/ejecting) the material pieces 101into one or more N (N≥1) sorting receptacles 136 . . . 139 according tothe determined classifications. Four sorting devices 126 . . . 129 andfour sorting receptacles 136 . . . 139 associated with the sortingdevices are illustrated in FIG. 1 as merely a non-limiting example.

As described herein, embodiments of the present disclosure areconfigured to identify live airbag modules within a moving stream ofscrap pieces (e.g., distinguish live airbag modules from otherautomotive scrap pieces), and to sort these live airbag modules so thatthey are removed/diverted/ejected from the conveyor system.

The sorting devices may include any well-known sorting mechanisms forremoving/diverting/ejecting selected material pieces 101 identified aslive airbag modules towards a desired location, including, but notlimited to, diverting the material pieces 101 from the conveyor beltsystem into one or more sorting receptacles. In accordance with certainembodiments of the present disclosure, the sorting mechanism forremoval/diversion/ejection of a live airbag module from a conveyorsystem may be configured so that it removes/diverts/ejects the airbagfrom the conveyor system regardless whether other material pieces withinthe vicinity of the live airbag module are also removed/diverted/ejectedfrom the conveyor system along with the live airbag module, since it maybe more important that the live airbag module beremoved/diverted/ejected even if it means the loss of one or more othermaterial pieces from the remaining scrap stream. For example, referringto any of FIG. 6A, 6B, or 7, it can be readily seen that there are otherscrap pieces within the vicinity of live airbag modules. In suchinstances, the other scrap pieces that are within the vicinity of aclassified live airbag module may be diverted from the conveyor beltinto the designated receptacle along with the classified live airbagmodule, since it is more important that the live airbag module isremoved from the stream of scrap pieces than to attempt to only divertthe classified live airbag module at the risk of not being accurateenough with the diverting action by the sorting mechanism, resulting inthe classified live airbag module not being removed from the stream ofscrap pieces.

Mechanisms that may be used to remove/divert/eject the material piecesinclude robotically removing the material pieces from the conveyor belt,pushing the material pieces from the conveyor belt (e.g., with paintbrush type plungers), causing an opening (e.g., a trap door) in theconveyor system 103 from which a material piece may drop, or using airjets to separate the material pieces into separate receptacles as theyfall from the edge of the conveyor belt. A pusher device, as that termis used herein, may refer to any form of device which may be activatedto dynamically displace an object on or from a conveyor system/device,employing pneumatic, mechanical, hydraulic, or vacuum actuators, orother means to do so, such as any appropriate type of mechanical pushingmechanism (e.g., an ACME screw drive), pneumatic pushing mechanism, orair jet pushing mechanism.

In accordance with certain embodiments of the present disclosure, thelive airbag modules may need to be removed/diverted/ejected from theconveyor system in a relatively “gentle” manner so that the live airbagmodules are not activated so that they inflate/explode. In accordancewith certain embodiments of the present disclosure, any technique forremoval/diversion/ejection of a live airbag module from a conveyorsystem may be utilized, wherein the force by which theremoval/diversion/ejection is performed is configured so that it doesnot result in an activation of the live airbag module so that itinflates or explodes. For example, the sorting may be performed by asorting mechanism that diverts the live airbag module into a receptacleusing a diverting force configured to not activate the live airbagmodule. Thus, the sorting mechanism can be configured so that it divertsthe live airbag module off of the conveyor belt with sufficient force tomove the live airbag module, but utilizing less force that it known tocause such live airbag modules to activate. This, of course, can bedetermined using trial an error. In accordance with certain embodimentsof the present disclosure, such a sorting mechanism may be a paint brushtype plunger.

Robotic removal may be performed by some sort of appropriate roboticarm, such as a Stewart Platform, a Delta Robot, or a multiple pronggripper.

In addition to the N sorting receptacles 136 . . . 139 into whichmaterial pieces 101 (e.g., live airbag modules) areremoved/diverted/ejected, the system 100 may also include a receptacle140 that receives material pieces 101 (e.g., the remaining automotivescrap pieces) not diverted/ejected from the conveyor system 103 into anyof the aforementioned sorting receptacles 136 . . . 139.

Depending upon the variety of classifications of material piecesdesired, multiple classifications may be mapped to a single sortingdevice and associated sorting receptacle. In other words, there need notbe a one-to-one correlation between classifications and sortingreceptacles. For example, it may be desired by the user to sort certainclassifications of materials (e.g., live airbag modules and othermaterial types) into the same sorting receptacle. To accomplish thissort, when a material piece 101 is classified as falling into apredetermined grouping of classifications, the same sorting device maybe activated to sort these into the same sorting receptacle. Suchcombination sorting may be applied to produce any desired combination ofsorted material pieces. The mapping of classifications may be programmedby the user (e.g., using the algorithm(s) operated by the computersystem 107) to produce such desired combinations. Additionally, theclassifications of material pieces are user-definable, and not limitedto any particular known classifications of material pieces.

The conveyor system 103 may include a circular conveyor (not shown) sothat unclassified material pieces are returned to the beginning of thesystem 100 and run through the system 100 again. Moreover, because thesystem 100 is able to specifically track each material piece 101 as ittravels on the conveyor system 103, some sort of sorting device (e.g.,the sorting device 129) may be implemented to remove/direct/eject amaterial piece 101 that the system 100 has failed to classify (e.g., amaterial piece that has not been classified as a live airbag moduleaccording to a predetermined threshold value but the user desires formaterial pieces with a live airbag module classification assigned acertain value below the predetermined threshold to be classified as alive airbag module nevertheless in order to have a higher probabilitythat all or substantially all of the live airbag modules areremoved/directed/ejected) after a predetermined number of cycles throughthe system 100 (or the material piece 101 is collected in receptacle140).

As exemplified in FIGS. 2A-2C, the systems and methods described hereinmay be applied to classify and/or sort individual airbag modules havingany of a variety of sizes.

As previously noted, certain embodiments of the present disclosure mayimplement one or more vision systems (e.g., vision system 110) in orderto identify, track, and/or classify material pieces. In accordance withembodiments of the present disclosure, such a vision system(s) mayoperate alone to identify and/or classify and sort material pieces, ormay operate in combination with a sensor system (e.g., sensor system120) to identify and/or classify and sort material pieces. If a system(e.g., system 100) is configured to operate solely with such a visionsystem(s) 110, then the sensor system 120 may be omitted from the system100 (or simply deactivated).

Such a vision system may be configured with one or more devices forcapturing or acquiring images of the material pieces as they pass by ona conveyor system. The devices may be configured to capture or acquireany desired range of wavelengths irradiated or reflected by the materialpieces, including, but not limited to, visible, infrared (“IR”),ultraviolet (“UV”) light. For example, the vision system may beconfigured with one or more cameras (still and/or video, either of whichmay be configured to capture two-dimensional, three-dimensional, and/orholographical images) positioned in proximity (e.g., above) the conveyorsystem so that images of the material pieces are captured as they passby the sensor system(s). In accordance with alternative embodiments ofthe present disclosure, data captured by a sensor system 120 may beprocessed (converted) into data to be utilized (either solely or incombination with the image data captured by the vision system 110) forclassifying/sorting of the material pieces. Such an implementation maybe in lieu of, or in combination with, utilizing the sensor system 120for classifying material pieces.

Regardless of the type(s) of sensed characteristics/information capturedof the material pieces, the information may then be sent to a computersystem (e.g., computer system 107) to be processed (e.g., by an AIsystem) in order to identify and/or classify material pieces. An AIsystem may implement any known AI system (e.g., Artificial NarrowIntelligence (“ANI”), Artificial General Intelligence (“AGI”), andArtificial Super Intelligence (“ASI”)) or derivation thereof yet to bedeveloped, a machine learning system including one that implements aneural network (e.g., artificial neural network, deep neural network,convolutional neural network, recurrent neural network, autoencoders,reinforcement learning, etc.), a machine learning system implementingsupervised learning, unsupervised learning, semi-supervised learning,reinforcement learning, self learning, feature learning, sparsedictionary learning, anomaly detection, robot learning, association rulelearning, fuzzy logic, deep learning algorithms, deep structuredlearning hierarchical learning algorithms, extreme learning machine,support vector machine (“SVM”) (e.g., linear SVM, nonlinear SVM, SVMregression, etc.), decision tree learning (e.g., classification andregression tree (“CART”), ensemble methods (e.g., ensemble learning,Random Forests, Bagging and Pasting, Patches and Subspaces, Boosting,Stacking, etc.), dimensionality reduction (e.g., Projection, ManifoldLearning, Principal Components Analysis, etc.), and/or deep machinelearning algorithms. Non-limiting examples of publicly available machinelearning software and libraries that could be utilized withinembodiments of the present disclosure include Python, OpenCV, Inception,Theano, Torch, PyTorch, Pylearn2, Numpy, Blocks, TensorFlow, MXNet,Caffe, Lasagne, Keras, Chainer, Matlab Deep Learning, CNTK, MatConvNet(a MATLAB toolbox implementing convolutional neural networks forcomputer vision applications), DeepLearnToolbox (a Matlab toolbox forDeep Learning (from Rasmus Berg Palm)), BigDL, Cuda-Convnet (a fastC++/CUDA implementation of convolutional (or more generally,feed-forward) neural networks), Deep Belief Networks, RNNLM,RNNLIB-RNNLIB, matrbm, deeplearning4j, Eblearn.lsh, deepmat, MShadow,Matplotlib, SciPy, CXXNET, Nengo-Nengo, Eblearn, cudamat, Gnumpy, 3-wayfactored RBM and mcRBM, mPoT (Python code using CUDAMat and Gnumpy totrain models of natural images), ConvNet, Elektronn, OpenNN,NeuralDesigner, Theano Generalized Hebbian Learning, Apache Singa,Lightnet, and SimpleDNN.

In accordance with embodiments of the present disclosure, identifyingand/or classifying each of the material pieces 101 may be performed byan AI system implementing semantic segmentation. However, otherimplementations may be utilized, such as image segmentation such as MaskR-CNN (e.g., with Python code), panoptic segmentation, instancesegmentation, block segmentation, or bounding box algorithms.

Image segmentation is capable of identifying/classifying material piecesthat are partially occluded by other material pieces. FIGS. 6A and 7show exemplary images of material pieces overlaying each other so thatone or more live airbag modules are partially occluded, but which can beidentified/classified as live airbag modules by embodiments of thepresent disclosure (as demonstrated in FIG. 7), and thus distinguishedfrom other automotive scrap pieces, such as when the AI systemimplements some form of image segmentation algorithm.

Configuring of an AI system often occurs in multiple stages. Forexample, first, training occurs, which may be performed offline in thatthe system 100 is not being utilized to perform actualclassifying/sorting of material pieces. The system 100 may be utilizedto train the AI system in that homogenous sets (also referred to hereinas control samples) of material pieces (i.e., having the same types orclasses of materials) may be passed through the system 100 (e.g., by aconveyor system 103); and all such material pieces may not be sorted,but may be collected in a common receptacle (e.g., receptacle 140).Alternatively, the training may be performed at another location remotefrom the system 100, including using some other mechanism for collectingsensed information (characteristics) of control sets of material pieces.During this training stage, algorithms within the AI system extractfeatures from the captured information (e.g., using image processingtechniques well known in the art). Non-limiting examples of trainingalgorithms include, but are not limited to, linear regression, gradientdescent, feed forward, polynomial regression, learning curves,regularized learning models, and logistic regression. Additionally,training may include data curation, data organization, data labeling,semi-synthetic data composition, synthetic data generation, dataaugmentation and other activity (e.g., off-machine training on separateequipment designed for that purpose, as well as “equipmentless” trainingdone entirely in computer memory (simulated, augmented, etc.)) aroundpreparation of the “curriculum” (e.g., the training or control sets)that is being taught to the AI system. It is during this training stagethat the algorithms within the AI system learn the relationships betweenmaterials and their features/characteristics (e.g., as captured by thevision system and/or sensor system(s)), creating a knowledge base forlater classification of a heterogeneous mixture of material piecesreceived by the system 100, which may then be sorted by desiredclassifications. Such a knowledge base may include one or morelibraries, wherein each library includes parameters (e.g., neuralnetwork parameters) for utilization by the AI system in classifyingmaterial pieces. For example, one particular library may includeparameters configured by the training stage to recognize and classifyairbag modules. In accordance with certain embodiments of the presentdisclosure, such libraries may be inputted into the AI system and thenthe user of the system 100 may be able to adjust certain ones of theparameters in order to adjust an operation of the system 100 (forexample, adjusting the threshold effectiveness of how well the AI systemidentifies/classifies, and distinguishes live airbag modules from amixture of materials (e.g., a moving stream of automotive scrappieces)).

As shown by the exemplary images in FIGS. 2A-2B, during the trainingstage, examples of one or more live airbag modules, which may bereferred to herein as a set of one or more control samples, may bedelivered past the vision system and/or one or more sensor systems(e.g., by a conveyor system) so that the algorithms within the AI systemdetect, extract, and learn what features represent such a type or classof material. For example, each of the live airbag modules are passedthrough such a training stage so that the algorithms within the AIsystem “learn” (are trained) how to detect, recognize, and classify liveairbag modules (see FIG. 2C). In the case of training a vision system(e.g., the vision system 110), trained to visually discern betweenmaterial pieces. This creates a library of parameters particular to liveairbag modules. Then, for example, the same process may be performedwith respect to a certain class, or type, of metal alloy (or a mixtureof automotive scrap pieces that are not live airbag modules), creating alibrary of parameters particular to that class, or type, of metal alloy,and so on. For each class or type of material to be classified by thesystem, any number of exemplary material pieces of that class or type ofmaterial may be passed by the vision system and/or one or more sensorsystem(s). Given a captured image or other captured characteristic asinput data, the AI algorithm(s) may use N classifiers, each of whichtest for one of N different material classes or types.

After the algorithms have been established and the AI system hassufficiently learned (been trained) the differences (e.g., visuallydiscernible differences) for the material classifications (e.g., withina user-defined level of statistical confidence), the libraries for thedifferent material classifications are then implemented into a materialclassifying/sorting system (e.g., system 100) to be used for identifyingand/or classifying material pieces (e.g., live airbag modules) from aheterogeneous mixture of material pieces (e.g., stream of automotivescrap pieces), and then sorting such classified material pieces.

Techniques to construct, optimize, and utilize an AI system are known tothose of ordinary skill in the art as found in relevant literature.Examples of such literature include the publications: Krizhevsky et al.,“ImageNet Classification with Deep Convolutional Networks,” Proceedingsof the 25th International Conference on Neural Information ProcessingSystems, Dec. 3-6, 2012, Lake Tahoe, Nev., and LeCun et al.,“Gradient-Based Learning Applied to Document Recognition,” Proceedingsof the IEEE, Institute of Electrical and Electronic Engineers (IEEE),November 1998, both of which are hereby incorporated by reference hereinin their entirety.

In an example technique, data captured by a vision or sensor system withrespect to a particular material piece (e.g., a live airbag module) maybe processed as an array of data values (within a data processing system(e.g., the data processing system 3400 of FIG. 10) implementing(configured with) an AI system). For example, the data may be spectraldata captured by a digital camera or other type of sensor system withrespect to a particular material piece and processed as an array of datavalues (e.g., image data packets). Each data value may be represented bya single number, or as a series of numbers representing values. Thesevalues may be multiplied by neuron weight parameters (e.g., with aneural network), and may possibly have a bias added. This may be fedinto a neuron nonlinearity. The resulting number output by the neuroncan be treated much as the values were, with this output multiplied bysubsequent neuron weight values, a bias optionally added, and once againfed into a neuron nonlinearity. Each such iteration of the process isknown as a “layer” of the neural network. The final outputs of the finallayer may be interpreted as probabilities that a material is present orabsent in the captured data pertaining to the material piece. Examplesof such a process are described in detail in both of the previouslynoted “ImageNet Classification with Deep Convolutional Networks” and“Gradient-Based Learning Applied to Document Recognition” references. Inaccordance with embodiments of the present disclosure, a bias may beconfigured so that the AI system classifies an automotive scrap piece asa live airbag module because a captured visual image of the automotivescrap piece contains a visual characteristic that results in thecaptured visual image to resemble a live airbag module. The bias may beconfigured so that a false positive occurs more than a false negative ina ratio greater than a predetermined threshold (e.g., 95%). A falsepositive is an instance where the classification results in identifyingan automotive scrap piece as a live airbag module when it is actuallynot (such as when an automotive scrap piece physically resembles a liveairbag module). A false negative is an instance where the classificationresults in a failure to identify a live airbag module. Since it isvitally important to remove the live airbag modules from the stream ofautomotive scrap pieces, it may be acceptable to have a very highpredetermined ratio of false positives to false negatives configuredinto the AI system's classification algorithms, even though this mayresult in the removal of other automotive scrap pieces from the streamof scrap pieces.

In accordance with certain embodiments of the present disclosure inwhich a neural network is implemented, as a final layer (the“classification layer”), the final set of neurons' output is trained torepresent the likelihood a material piece (e.g., an airbag module) isassociated with the captured data. During operation, if the likelihoodthat a material piece is associated with the captured data is over auser-specified threshold, then it is determined that the particularmaterial piece is indeed associated with the captured data. Thesetechniques can be extended to determine not only the presence of a typeof material associated with particular captured data, but also whethersub-regions of the particular captured data belong to one type ofmaterial or another type of material. This process is known assegmentation, and techniques to use neural networks exist in theliterature, such as those known as “fully convolutional” neuralnetworks, or networks that otherwise include a convolutional portion(i.e., are partially convolutional), if not fully convolutional. Thisallows for material location and size to be determined. Examples includeMask R-CNN implementing image segmentation.

It should be understood that the present disclosure is not exclusivelylimited to AI techniques. Other common techniques for materialclassification/identification may also be used. For instance, a sensorsystem may utilize optical spectrometric techniques using multi- orhyper-spectral cameras to provide a signal that may indicate thepresence or absence of a type of material (e.g., containing one or moreparticular elements) by examining the spectral emissions (i.e., spectralimaging) of the material. Spectral images of a material piece (e.g., anairbag module) may also be used in a template-matching algorithm,wherein a database of spectral images is compared against an acquiredspectral image to find the presence or absence of certain types ofmaterials from that database. A histogram of the captured spectral imagemay also be compared against a database of histograms. Similarly, a bagof words model may be used with a feature extraction technique, such asscale-invariant feature transform (“SIFT”), to compare extractedfeatures between a captured image and those in a database. In accordancewith certain embodiments of the present disclosure, instead of utilizinga training stage whereby control samples of material pieces are passedby the vision system and/or sensor system(s), training of the machinelearning system may be performed utilizing a labeling/annotationtechnique (or any other supervised learning technique) whereby asdata/information of material pieces are captured by a vision/sensorsystem, a user inputs a label or annotation that identifies eachmaterial piece (e.g., a live airbag module), which is then used tocreate the library for use by the machine learning system whenclassifying material pieces within a heterogenous mixture of materialpieces. In other words, a previously generated knowledge base ofcharacteristics captured from one or more samples of a class ofmaterials may be accomplished by any of the techniques disclosed herein,whereby such a knowledge base is then utilized to automatically classifymaterials.

Therefore, as disclosed herein, certain embodiments of the presentdisclosure provide for the identification/classification of one or moredifferent types or classes of materials in order to determine whichmaterial pieces (e.g., live airbag modules) should be diverted from aconveyor system in defined groups. In accordance with certainembodiments, AI techniques are utilized to train (i.e., configure) aneural network to identify a variety of one or more different classes ortypes of materials. Spectral images, or other types of sensedinformation, are captured of materials (e.g., traveling on a conveyorsystem), and based on the identification/classification of suchmaterials, the systems described herein can decide which material pieceshould be allowed to remain on the conveyor system, and which should bediverted/removed from the conveyor system (for example, either into acollection receptacle, or diverted onto another conveyor system).

One point of mention here is that, in accordance with certainembodiments of the present disclosure, thecollected/captured/detected/extracted features/characteristics (e.g.,spectral images) of the material pieces may not be necessarily simplyparticularly identifiable or discernible physical characteristics; theycan be abstract formulations that can only be expressed mathematically,or not mathematically at all; nevertheless, the AI system may beconfigured to parse the spectral data to look for patterns that allowthe control samples to be classified during the training stage.Furthermore, the machine learning system may take subsections ofcaptured information (e.g., spectral images) of a material piece andattempt to find correlations between the pre-defined classifications.

In accordance with certain embodiments of the present disclosure,instead of utilizing a training stage whereby control samples ofmaterial pieces are passed by the vision system and/or sensor system(s),training of the AI system may be performed utilizing alabeling/annotation technique (or any other supervised learningtechnique) whereby as data/information of material pieces (e.g., liveairbag modules) are captured by a vision/sensor system, a user inputs alabel or annotation that identifies each material piece, which is thenused to create the library for use by the AI system when classifyingmaterial pieces within a heterogenous mixture of material pieces.

In accordance with certain embodiments of the present disclosure, anysensed characteristics output by any of the sensor systems 120 disclosedherein may be input into an AI system in order to classify and/or sortmaterials. For example, in an AI system implementing supervisedlearning, sensor system 120 outputs that uniquely characterize aparticular type or composition of material (e.g., live airbag modules)may be used to train the AI system.

FIG. 3 illustrates a flowchart diagram depicting exemplary embodimentsof a process 3500 of classifying/sorting material pieces utilizing avision system and/or one or more sensor systems in accordance withcertain embodiments of the present disclosure. The process 3500 may beconfigured to operate within any of the embodiments of the presentdisclosure described herein, including the system 100 of FIG. 1.Operation of the process 3500 may be performed by hardware and/orsoftware, including within a computer system (e.g., computer system 3400of FIG. 5) controlling the system (e.g., the computer system 107, thevision system 110, and/or the sensor system(s) 120 of FIG. 1). In theprocess block 3501, the material pieces (e.g., mixture of automotivescrap pieces) may be deposited onto a conveyor system, such asrepresented in FIGS. 6A and 6B. In the process block 3502, the locationon the conveyor system of each material piece is detected for trackingof each material piece as it travels through the system 100. This may beperformed by the vision system 110 (for example, by distinguishing amaterial piece from the underlying conveyor system material while incommunication with a conveyor system position detector (e.g., theposition detector 105)). Alternatively, a material piece tracking device111 can be used to track the pieces. Or, any system that can create alight source (including, but not limited to, visual light, UV, and IR)and have a detector that can be used to locate the pieces. In theprocess block 3503, when a material piece has traveled in proximity toone or more of the vision system and/or the sensor system(s), sensedinformation/characteristics of the material piece is captured/acquired.In the process block 3504, a vision system (e.g., implemented within thecomputer system 107), such as previously disclosed, may performpre-processing of the captured information, which may be utilized todetect (extract) information of each of the material pieces (e.g., fromthe background (e.g., the conveyor belt); in other words, thepre-processing may be utilized to identify the difference between thematerial piece and the background). Well-known image processingtechniques such as dilation, thresholding, and contouring may beutilized to identify the material piece as being distinct from thebackground. In the process block 3505, segmentation may be performed.For example, the captured information may include information pertainingto one or more material pieces. Additionally, a particular materialpiece may be located on a seam of the conveyor belt when its image iscaptured. Therefore, it may be desired in such instances to isolate theimage of an individual material piece from the background of the image.In an exemplary technique for the process block 3505, a first step is toapply a high contrast of the image; in this fashion, background pixelsare reduced to substantially all black pixels, and at least some of thepixels pertaining to the material piece are brightened to substantiallyall white pixels. The image pixels of the material piece that are whiteare then dilated to cover the entire size of the material piece. Afterthis step, the location of the material piece is a high contrast imageof all white pixels on a black background. Then, a contouring algorithmcan be utilized to detect boundaries of the material piece. The boundaryinformation is saved, and the boundary locations are then transferred tothe original image. Segmentation is then performed on the original imageon an area greater than the boundary that was earlier defined. In thisfashion, the material piece is identified and separated from thebackground.

In accordance with embodiments of the present disclosure, the processblock 3505 may implement a semantic segmentation process, whichidentifies the airbag modules within a heterogeneous mixture of materialpieces, such as represented in FIG. 7. Alternatively, instancesegmentation, such as Mask R-CNN, or panoptic segmentation may beutilized.

In the optional process block 3506, the material pieces may be conveyedalong the conveyor system within proximity of a material piece trackingdevice and/or a sensor system in order to track each of the materialpieces and/or determine a size and/or shape of the material pieces,which may be useful if an XRF system or some other spectroscopy sensoris also implemented within the sorting system. In the process block3507, post processing may be performed. Post processing may involveresizing the captured information/data to prepare it for use in theneural networks. This may also include modifying certain properties(e.g., enhancing image contrast, changing the image background, orapplying filters) in a manner that will yield an enhancement to thecapability of the AI system to classify and distinguish the materialpieces. In the process block 3509, the data may be resized. Dataresizing may be desired under certain circumstances to match the datainput requirements for certain AI systems, such as neural networks. Forexample, neural networks may require much smaller image sizes (e.g.,225×255 pixels or 299×299 pixels) than the sizes of the images capturedby typical digital cameras. Moreover, the smaller the input data size,the less processing time is needed to perform the classification. Thus,smaller data sizes can ultimately increase the throughput of the system100 and increase its value.

In the process blocks 3510 and 3511, each material piece isidentified/classified based on the sensed/detected features. Forexample, the process block 3510 may be configured with a neural networkemploying one or more algorithms, which compare the extracted featureswith those stored in a previously generated knowledge base (e.g.,generated during a training stage), and assigns the classification withthe highest match to each of the material pieces based on such acomparison. The algorithms may process the captured information/data ina hierarchical manner by using automatically trained filters. The filterresponses are then successfully combined in the next levels of thealgorithms until a probability is obtained in the final step. In theprocess block 3511, these probabilities may be used for each of the Nclassifications to decide into which of the N sorting receptacles therespective material pieces should be sorted. For example, each of the Nclassifications may be assigned to one sorting receptacle, and thematerial piece under consideration is sorted into that receptacle thatcorresponds to the classification returning the highest probabilitylarger than a predefined threshold. Within embodiments of the presentdisclosure, such predefined thresholds may be preset by the user (e.g.,to ensure that false positive classifications substantially outnumberfalse negative classifications). A particular material piece may besorted into an outlier receptacle (e.g., sorting receptacle 140) if noneof the probabilities is larger than the predetermined threshold.

Next, in the process block 3512, a sorting device is activatedcorresponding to the classification, or classifications, of the materialpiece (e.g., instructions sent to the sorting device to sort). Betweenthe time at which the image of the material piece was captured and thetime at which the sorting device is activated, the material piece hasmoved from the proximity of the vision system and/or sensor system(s) toa location downstream on the conveyor system (e.g., at the rate ofconveying of a conveyor system). In embodiments of the presentdisclosure, the activation of the sorting device is timed such that asthe material piece passes the sorting device mapped to theclassification of the material piece, the sorting device is activated,and the material piece is removed/diverted/ejected from the conveyorsystem (e.g., into its associated sorting receptacle). Withinembodiments of the present disclosure, the activation of a sortingdevice may be timed by a respective position detector that detects whena material piece is passing before the sorting device and sends a signalto enable the activation of the sorting device. In the process block3513, the sorting receptacle corresponding to the sorting device thatwas activated receives the removed/diverted/ejected material piece.

FIG. 4 illustrates a flowchart diagram depicting exemplary embodimentsof a process 400 of sorting material pieces in accordance with certainembodiments of the present disclosure. The process 400 may be configuredto operate within any of the embodiments of the present disclosuredescribed herein, including the system 100 of FIG. 1. The process 400may be configured to operate in conjunction with the process 3500. Forexample, in accordance with certain embodiments of the presentdisclosure, the process blocks 403 and 404 may be incorporated in theprocess 3500 (e.g., operating in series or in parallel with the processblocks 3503-3510) in order to combine the efforts of a vision system 110that is implemented in conjunction with an AI system with a sensorsystem (e.g., the sensor system 120) that is not implemented inconjunction with an AI system in order to classify and/or sort materialpieces.

Operation of the process 400 may be performed by hardware and/orsoftware, including within a computer system (e.g., computer system 3400of FIG. 5) controlling the system (e.g., the computer system 107 of FIG.1). In the process block 401, the material pieces may be deposited ontoa conveyor system. Next, in the optional process block 402, the materialpieces may be conveyed along the conveyor system within proximity of amaterial piece tracking device and/or an optical imaging system in orderto track each material piece and/or determine a size and/or shape of thematerial pieces. In the process block 403, when a material piece hastraveled in proximity of the sensor system, the material piece may beinterrogated, or stimulated, with EM energy (waves) or some other typeof stimulus appropriate for the particular type of sensor technologyutilized by the sensor system. In the process block 404, physicalcharacteristics of the material piece are sensed/detected and capturedby the sensor system. In the process block 405, for at least some of thematerial pieces, the type of material is identified/classified based (atleast in part) on the captured characteristics, which may be combinedwith the classification by the AI system in conjunction with the visionsystem 110.

Next, if sorting of the material pieces is to be performed, in theprocess block 406, a sorting device corresponding to the classification,or classifications, of the material piece is activated. Between the timeat which the material piece was sensed and the time at which the sortingdevice is activated, the material piece has moved from the proximity ofthe sensor system to a location downstream on the conveyor system, atthe rate of conveying of the conveyor system. In certain embodiments ofthe present disclosure, the activation of the sorting device is timedsuch that as the material piece passes the sorting device mapped to theclassification of the material piece, the sorting device is activated,and the material piece is removed/diverted/ejected from the conveyorsystem into its associated sorting receptacle. Within certainembodiments of the present disclosure, the activation of a sortingdevice may be timed by a respective position detector that detects whena material piece is passing before the sorting device and sends a signalto enable the activation of the sorting device. In the process block407, the sorting receptacle corresponding to the sorting device that wasactivated receives the removed/diverted/ejected material piece.

In accordance with certain embodiments of the present disclosure, aplurality of at least a portion of the system 100 may be linked togetherin succession in order to perform multiple iterations or layers ofsorting. For example, when two or more systems 100 are linked in such amanner, the conveyor system may be implemented with a single conveyorbelt, or multiple conveyor belts, conveying the material pieces past afirst vision system (and, in accordance with certain embodiments, asensor system) configured for sorting material pieces of a first set ofa heterogeneous mixture of materials by a sorter (e.g., the firstautomation control system 108 and associated one or more sorting devices126 . . . 129) into a first set of one or more receptacles (e.g.,sorting receptacles 136 . . . 139), and then conveying the materialpieces past a second vision system (and, in accordance with certainembodiments, another sensor system) configured for sorting materialpieces of a second set of a heterogeneous mixture of materials by asecond sorter into a second set of one or more sorting receptacles. Forexample, the first sorting system may sort out live airbag modules sothat they are safely removed from the stream of automotive scrap piecesbefore the second sorting system sorts between two or more metal alloys.A further discussion of such multistage sorting is in U.S. publishedpatent application no. 2022/0016675, which is hereby incorporated byreference herein.

Such successions of systems 100 can contain any number of such systemslinked together in such a manner. In accordance with certain embodimentsof the present disclosure, each successive vision system may beconfigured to sort out a different classified or type of material thanthe previous system(s).

In accordance with various embodiments of the present disclosure,different types or classes of materials may be classified by differenttypes of sensors each for use with an AI system, and combined toclassify material pieces in a stream of scrap or waste.

In accordance with various embodiments of the present disclosure, data(e.g., spectral data) from two or more sensors can be combined using asingle or multiple AI systems to perform classifications of materialpieces.

In accordance with various embodiments of the present disclosure,multiple sensor systems can be mounted onto a single conveyor system,with each sensor system utilizing a different AI system. In accordancewith various embodiments of the present disclosure, multiple sensorsystems can be mounted onto different conveyor systems, with each sensorsystem utilizing a different AI system.

With reference now to FIG. 5, a block diagram illustrating a dataprocessing (“computer”) system 3400 is depicted in which aspects ofembodiments of the disclosure may be implemented. (The terms “computer,”“system,” “computer system,” and “data processing system” may be usedinterchangeably herein.) The computer system 107, the automation controlsystem 108, aspects of the sensor system(s) 120, and/or the visionsystem 110 may be configured similarly as the computer system 3400. Thecomputer system 3400 may employ a local bus 3405 (e.g., a peripheralcomponent interconnect (“PCI”) local bus architecture). Any suitable busarchitecture may be utilized such as Accelerated Graphics Port (“AGP”)and Industry Standard Architecture (“ISA”), among others. One or moreprocessors 3415, volatile memory 3420, and non-volatile memory 3435 maybe connected to the local bus 3405 (e.g., through a PCI Bridge (notshown)). An integrated memory controller and cache memory may be coupledto the one or more processors 3415. The one or more processors 3415 mayinclude one or more central processor units and/or one or more graphicsprocessor units and/or one or more tensor processing units. Additionalconnections to the local bus 3405 may be made through direct componentinterconnection or through add-in boards. In the depicted example, acommunication (e.g., network (LAN)) adapter 3425, an I/O (e.g., smallcomputer system interface (“SCSI”) host bus) adapter 3430, and expansionbus interface (not shown) may be connected to the local bus 3405 bydirect component connection. An audio adapter (not shown), a graphicsadapter (not shown), and display adapter 3416 (coupled to a display3440) may be connected to the local bus 3405 (e.g., by add-in boardsinserted into expansion slots).

The user interface adapter 3412 may provide a connection for a keyboard3413 and a mouse 3414, modem/router (not shown), and additional memory(not shown). The I/O adapter 3430 may provide a connection for a harddisk drive 3431, a tape drive 3432, and a CD-ROM drive (not shown).

One or more operating systems may be run on the one or more processors3415 and used to coordinate and provide control of various componentswithin the computer system 3400. In FIG. 5, the operating system(s) maybe a commercially available operating system. An object-orientedprogramming system (e.g., Java, Python, etc.) may run in conjunctionwith the operating system and provide calls to the operating system fromprograms or programs (e.g., Java, Python, etc.) executing on the system3400. Instructions for the operating system, the object-orientedoperating system, and programs may be located on non-volatile memory3435 storage devices, such as a hard disk drive 3431, and may be loadedinto volatile memory 3420 for execution by the processor 3415.

Those of ordinary skill in the art will appreciate that the hardware inFIG. 5 may vary depending on the implementation. Other internal hardwareor peripheral devices, such as flash ROM (or equivalent nonvolatilememory) or optical disk drives and the like, may be used in addition toor in place of the hardware depicted in FIG. 5. Also, any of theprocesses of the present disclosure may be applied to a multiprocessorcomputer system, or performed by a plurality of such systems 3400. Forexample, training of the vision system 110 may be performed by a firstcomputer system 3400, while operation of the vision system 110 forclassifying may be performed by a second computer system 3400.

As another example, the computer system 3400 may be a stand-alone systemconfigured to be bootable without relying on some type of networkcommunication interface, whether or not the computer system 3400includes some type of network communication interface. As a furtherexample, the computer system 3400 may be an embedded controller, whichis configured with ROM and/or flash ROM providing non-volatile memorystoring operating system files or user-generated data.

The depicted example in FIG. 5 and above-described examples are notmeant to imply architectural limitations. Further, a computer programform of aspects of the present disclosure may reside on any computerreadable storage medium (i.e., floppy disk, compact disk, hard disk,tape, ROM, RAM, etc.) used by a computer system.

As has been described herein, embodiments of the present disclosure maybe implemented to perform the various functions described foridentifying, tracking, classifying, and/or sorting material pieces. Suchfunctionalities may be implemented within hardware and/or software, suchas within one or more data processing systems (e.g., the data processingsystem 3400 of FIG. 5), such as the previously noted computer system107, the vision system 110, aspects of the sensor system(s) 120, and/orthe automation control system 108. Nevertheless, the functionalitiesdescribed herein are not to be limited for implementation into anyparticular hardware/software platform.

As will be appreciated by one skilled in the art, aspects of the presentdisclosure may be embodied as a system, process, method, and/or programproduct. Accordingly, various aspects of the present disclosure may takethe form of an entirely hardware embodiment, an entirely softwareembodiment (including firmware, resident software, micro-code, etc.), orembodiments combining software and hardware aspects, which may generallybe referred to herein as a “circuit,” “circuitry,” “module,” or“system.” Furthermore, aspects of the present disclosure may take theform of a program product embodied in one or more computer readablestorage medium(s) having computer readable program code embodiedthereon. (However, any combination of one or more computer readablemedium(s) may be utilized. The computer readable medium may be acomputer readable signal medium or a computer readable storage medium.)

A computer readable storage medium may be, for example, but not limitedto, an electronic, magnetic, optical, electromagnetic, infrared,biologic, atomic, or semiconductor system, apparatus, controller, ordevice, or any suitable combination of the foregoing, wherein thecomputer readable storage medium is not a transitory signal per se. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium may include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (“RAM”) (e.g., RAM 3420 of FIG. 5), a read-onlymemory (“ROM”) (e.g., ROM 3435 of FIG. 5), an erasable programmableread-only memory (“EPROM” or flash memory), an optical fiber, a portablecompact disc read-only memory (“CD-ROM”), an optical storage device, amagnetic storage device (e.g., hard drive 3431 of FIG. 5), or anysuitable combination of the foregoing. In the context of this document,a computer readable storage medium may be any tangible medium that cancontain or store a program for use by or in connection with aninstruction execution system, apparatus, controller, or device. Programcode embodied on a computer readable signal medium may be transmittedusing any appropriate medium, including but not limited to wireless,wire line, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, controller, or device.

The flowchart and block diagrams in the figures illustrate architecture,functionality, and operation of possible implementations of systems,methods, processes, and program products according to variousembodiments of the present disclosure. In this regard, each block in theflowcharts or block diagrams may represent a module, segment, or portionof code, which includes one or more executable program instructions forimplementing the specified logical function(s). It should also be notedthat, in some implementations, the functions noted in the blocks mayoccur out of the order noted in the figures. For example, two blocksshown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved.

Modules implemented in software for execution by various types ofprocessors (e.g., GPU 3401, CPU 3415) may, for instance, include one ormore physical or logical blocks of computer instructions, which may, forinstance, be organized as an object, procedure, or function.Nevertheless, the executables of an identified module need not bephysically located together, but may include disparate instructionsstored in different locations which, when joined logically together,include the module and achieve the stated purpose for the module.Indeed, a module of executable code may be a single instruction, or manyinstructions, and may even be distributed over several different codesegments, among different programs, and across several memory devices.Similarly, operational data (e.g., material classification librariesdescribed herein) may be identified and illustrated herein withinmodules, and may be embodied in any suitable form and organized withinany suitable type of data structure. The operational data may becollected as a single data set, or may be distributed over differentlocations including over different storage devices. The data may provideelectronic signals on a system or network.

These program instructions may be provided to one or more processorsand/or controller(s) of a general purpose computer, special purposecomputer, or other programmable data processing apparatus (e.g.,controller) to produce a machine, such that the instructions, whichexecute via the processor(s) (e.g., GPU 3401, CPU 3415) of the computeror other programmable data processing apparatus, create circuitry ormeans for implementing the functions/acts specified in the flowchartand/or block diagram block or blocks.

It will also be noted that each block of the block diagrams and/orflowchart illustrations, and combinations of blocks in the blockdiagrams and/or flowchart illustrations, can be implemented by specialpurpose hardware-based systems (e.g., which may include one or moregraphics processing units (e.g., GPU 3401)) that perform the specifiedfunctions or acts, or combinations of special purpose hardware andcomputer instructions. For example, a module may be implemented as ahardware circuit including custom VLSI circuits or gate arrays,off-the-shelf semiconductors such as logic chips, transistors,controllers, or other discrete components. A module may also beimplemented in programmable hardware devices such as field programmablegate arrays, programmable array logic, programmable logic devices, orthe like.

In the description herein, a flow-charted technique may be described ina series of sequential actions. The sequence of the actions, and theelement performing the actions, may be freely changed without departingfrom the scope of the teachings. Actions may be added, deleted, oraltered in several ways. Similarly, the actions may be re-ordered orlooped. Further, although processes, methods, algorithms, or the likemay be described in a sequential order, such processes, methods,algorithms, or any combination thereof may be operable to be performedin alternative orders. Further, some actions within a process, method,or algorithm may be performed simultaneously during at least a point intime (e.g., actions performed in parallel), and can also be performed inwhole, in part, or any combination thereof.

Reference may be made herein to a device, circuit, circuitry, system, ormodule “configured to” perform a particular function or functions. Itshould be understood that this may include selecting predefined logicblocks and logically associating them, such that they provide particularlogic functions, which includes monitoring or control functions. It mayalso include programming computer software-based logic, wiring discretehardware components, or a combination of any or all of the foregoing.

To the extent not described herein, many details regarding specificmaterials, processing acts, and circuits are conventional, and may befound in textbooks and other sources within the computing, electronics,and software arts.

Computer program code, i.e., instructions, for carrying out operationsfor aspects of the present disclosure may be written in any combinationof one or more programming languages, including an object-orientedprogramming language such as Java, Smalltalk, Python, C++, or the like,conventional procedural programming languages, such as the “C”programming language or similar programming languages, programminglanguages such as MATLAB or LabVIEW, or any of the AI software disclosedherein. The program code may execute entirely on the user's computersystem, partly on the user's computer system, as a stand-alone softwarepackage, partly on the user's computer system (e.g., the computer systemutilized for sorting) and partly on a remote computer system (e.g., thecomputer system utilized to train the AI system), or entirely on theremote computer system or server. In the latter scenario, the remotecomputer system may be connected to the user's computer system throughany type of network, including a local area network (“LAN”) or a widearea network (“WAN”), or the connection may be made to an externalcomputer system (for example, through the Internet using an InternetService Provider). As an example of the foregoing, various aspects ofthe present disclosure may be configured to execute on one or more ofthe computer system 107, automation control system 108, the visionsystem 110, and aspects of the sensor system(s) 120.

These program instructions may also be stored in a computer readablestorage medium that can direct a computer system, other programmabledata processing apparatus, controller, or other devices to function in aparticular manner, such that the instructions stored in the computerreadable medium produce an article of manufacture including instructionswhich implement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The program instructions may also be loaded onto a computer, otherprogrammable data processing apparatus, controller, or other devices tocause a series of operational steps to be performed on the computer,other programmable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

One or more databases may be included in a host for storing andproviding access to data for the various implementations. One skilled inthe art will also appreciate that, for security reasons, any databases,systems, or components of the present disclosure may include anycombination of databases or components at a single location or atmultiple locations, wherein each database or system may include any ofvarious suitable security features, such as firewalls, access codes,encryption, de-encryption and the like. The database may be any type ofdatabase, such as relational, hierarchical, object-oriented, and/or thelike. Common database products that may be used to implement thedatabases include DB2 by IBM, any of the database products availablefrom Oracle Corporation, Microsoft Access by Microsoft Corporation, orany other database product. The database may be organized in anysuitable manner, including as data tables or lookup tables.

Association of certain data (e.g., for each of the material piecesprocessed by a material handling system described herein) may beaccomplished through any data association technique known and practicedin the art. For example, the association may be accomplished eithermanually or automatically. Automatic association techniques may include,for example, a database search, a database merge, GREP, AGREP, SQL,and/or the like. The association step may be accomplished by a databasemerge function, for example, using a key field in each of themanufacturer and retailer data tables. A key field partitions thedatabase according to the high-level class of objects defined by the keyfield. For example, a certain class may be designated as a key field inboth the first data table and the second data table, and the two datatables may then be merged on the basis of the class data in the keyfield. In these embodiments, the data corresponding to the key field ineach of the merged data tables is preferably the same. However, datatables having similar, though not identical, data in the key fields mayalso be merged by using AGREP, for example.

Aspects of the present disclosure provide a method of sorting liveairbag modules from a moving stream of automotive scrap pieces, whereinthe method includes conveying automotive scrap pieces past a visionsystem, wherein the automotive scrap pieces include a live airbagmodule; capturing visual images of the automotive scrap pieces;processing the captured visual images of the automotive scrap piecesthrough an artificial intelligence system in order to distinguish thelive airbag module from the other automotive scrap pieces; and sortingthe live airbag module from the moving stream of automotive scrappieces. The sorting may include diverting the live airbag module into areceptacle along with other automotive scrap pieces that are within avicinity of the live airbag module. The sorting may be performed withoutactivating the live airbag module. The sorting may be performed by asorting mechanism that diverts the live airbag module using a divertingforce configured to not activate the live airbag module. The sortingmechanism may be a paint brush type plunger. The live airbag module maybe partially occluded by at least one other automotive scrap piece sothat the vision system is unable to acquire spectral image data of anentirety of the live airbag module. The artificial intelligence systemmay be configured to identify the partially occluded live airbag module.The artificial intelligence system may be configured with a semanticsegmentation algorithm for distinguishing between live airbag modulesand other automotive scrap pieces. The method may further includesorting the automotive scrap pieces into separate metal alloys after thesorting of the live airbag modules from the stream of automotive scrappieces. The artificial intelligence system may be configured to classifya particular automotive scrap piece as a live airbag module in a ratioof false positives to false negatives greater than a predeterminedthreshold.

Aspects of the present disclosure provide a system for sorting liveairbag modules from a moving stream of automotive scrap pieces, whereinthe system includes a conveyor system for conveying automotive scrappieces past a vision system, wherein the automotive scrap pieces includea live airbag module; the vision system configured to capture visualimages of the automotive scrap pieces; a data processing systemconfigured with an artificial intelligence system configured to processthe captured visual images of the automotive scrap pieces through theartificial intelligence system in order to distinguish the live airbagmodule from the other automotive scrap pieces; and a sorting device forsorting the live airbag module from the moving stream of automotivescrap pieces. The sorting may include diverting the live airbag moduleinto a receptacle along with automotive scrap pieces that are within avicinity of the live airbag module. The sorting may be performed withoutactivating the live airbag module. The sorting device may include asorting mechanism that diverts the live airbag module using a divertingforce configured to not activate the live airbag module. The sortingmechanism may be a paint brush type plunger. The live airbag module maybe partially occluded by at least one other automotive scrap piece sothat the vision system is unable to acquire spectral image data of anentirety of the live airbag module. The artificial intelligence systemmay be configured to identify the partially occluded live airbag moduleand distinguish the partially occluded live airbag module from the otherautomotive scrap pieces. The artificial intelligence system may beconfigured with a Mask R-CNN algorithm for distinguishing between liveairbag modules and other automotive scrap pieces. The artificialintelligence system may be configured to classify a particularautomotive scrap piece as a live airbag module if the particularautomotive scrap piece sufficiently resembles a live airbag module. Theartificial intelligence system may be configured to classify aparticular automotive scrap piece as a live airbag module in a ratio offalse positives to false negatives greater than a predeterminedthreshold.

In the descriptions herein, numerous specific details are provided, suchas examples of programming, software modules, user selections, networktransactions, database queries, database structures, hardware modules,hardware circuits, hardware chips, controllers, etc., to provide athorough understanding of embodiments of the disclosure. One skilled inthe relevant art will recognize, however, that the disclosure may bepracticed without one or more of the specific details, or with othermethods, components, materials, and so forth. In other instances,well-known structures, materials, or operations may be not shown ordescribed in detail to avoid obscuring aspects of the disclosure.

Reference throughout this specification to “an embodiment,”“embodiments,” or similar language means that a particular feature,structure, or characteristic described in connection with theembodiments is included in at least one embodiment of the presentdisclosure. Thus, appearances of the phrases “in one embodiment,” “in anembodiment,” “embodiments,” “certain embodiments,” “variousembodiments,” and similar language throughout this specification may,but do not necessarily, all refer to the same embodiment. Furthermore,the described features, structures, aspects, and/or characteristics ofthe disclosure may be combined in any suitable manner in one or moreembodiments. Correspondingly, even if features may be initially claimedas acting in certain combinations, one or more features from a claimedcombination can in some cases be excised from the combination, and theclaimed combination can be directed to a sub-combination or variation ofa sub-combination.

Benefits, advantages, and solutions to problems have been describedabove with regard to specific embodiments. However, the benefits,advantages, solutions to problems, and any element(s) that may cause anybenefit, advantage, or solution to occur or become more pronounced maybe not to be construed as critical, required, or essential features orelements of any or all the claims. Further, no component describedherein is required for the practice of the disclosure unless expresslydescribed as essential or critical.

Herein, the term “or” may be intended to be inclusive, wherein “A or B”includes A or B and also includes both A and B. As used herein, the term“and/or” when used in the context of a listing of entities, refers tothe entities being present singly or in combination. Thus, for example,the phrase “A, B, C, and/or D” includes A, B, C, and D individually, butalso includes any and all combinations and subcombinations of A, B, C,and D.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the disclosure.As used herein, the singular forms “a,” “an,” and “the” may be intendedto include the plural forms as well, unless the context clearlyindicates otherwise.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below may be intendedto include any structure, material, or act for performing the functionin combination with other claimed elements as specifically claimed.

As used herein with respect to an identified property or circumstance,“substantially” refers to a degree of deviation that is sufficientlysmall so as to not measurably detract from the identified property orcircumstance. The exact degree of deviation allowable may in some casesdepend on the specific context.

As used herein, a plurality of items, structural elements, compositionalelements, and/or materials may be presented in a common list forconvenience. However, these lists should be construed as though eachmember of the list is individually identified as a separate and uniquemember. Thus, no individual member of such list should be construed as adefacto equivalent of any other member of the same list solely based ontheir presentation in a common group without indications to thecontrary.

Unless defined otherwise, all technical and scientific terms (such asacronyms used for chemical elements within the periodic table) usedherein have the same meaning as commonly understood to one of ordinaryskill in the art to which the presently disclosed subject matterbelongs. Although any methods, devices, and materials similar orequivalent to those described herein can be used in the practice ortesting of the presently disclosed subject matter, representativemethods, devices, and materials are now described.

The term “coupled,” as used herein, is not intended to be limited to adirect coupling or a mechanical coupling. Unless stated otherwise, termssuch as “first” and “second” are used to arbitrarily distinguish betweenthe elements such terms describe. Thus, these terms are not necessarilyintended to indicate temporal or other prioritization of such elements.

What is claimed is:
 1. A method of sorting live airbag modules from amoving stream of automotive scrap pieces, comprising: conveyingautomotive scrap pieces past a vision system, wherein the automotivescrap pieces include a live airbag module; capturing visual images ofthe automotive scrap pieces; processing the captured visual images ofthe automotive scrap pieces through an artificial intelligence system inorder to distinguish the live airbag module from the other automotivescrap pieces; and sorting the live airbag module from the moving streamof automotive scrap pieces.
 2. The method as recited in claim 1, whereinthe sorting includes diverting the live airbag module into a receptaclealong with other automotive scrap pieces that are within a vicinity ofthe live airbag module.
 3. The method as recited in claim 1, wherein thesorting is performed without activating the live airbag module.
 4. Themethod as recited in claim 1, wherein the sorting is performed by asorting mechanism that diverts the live airbag module using a divertingforce configured to not activate the live airbag module.
 5. The methodas recited in claim 4, wherein the sorting mechanism is a paint brushtype plunger.
 6. The method has recited in claim 2, wherein the liveairbag module is partially occluded by at least one other automotivescrap piece so that the vision system is unable to acquire spectralimage data of an entirety of the live airbag module.
 7. The method asrecited in claim 6, wherein the artificial intelligence system isconfigured to identify the partially occluded live airbag module.
 8. Themethod as recited in claim 7, wherein the artificial intelligence systemis configured with a semantic segmentation algorithm for distinguishingbetween live airbag modules and other automotive scrap pieces.
 9. Themethod as recited in claim 1, further comprising sorting the automotivescrap pieces into separate metal alloys after the sorting of the liveairbag modules from the stream of automotive scrap pieces.
 10. Themethod as recited in claim 1, wherein the artificial intelligence systemis configured to classify a particular automotive scrap piece as a liveairbag module in a ratio of false positives to false negatives greaterthan a predetermined threshold.
 11. A system for sorting live airbagmodules from a moving stream of automotive scrap pieces, comprising: aconveyor system for conveying automotive scrap pieces past a visionsystem, wherein the automotive scrap pieces include a live airbagmodule; the vision system configured to capture visual images of theautomotive scrap pieces; a data processing system configured with anartificial intelligence system configured to process the captured visualimages of the automotive scrap pieces through the artificialintelligence system in order to distinguish the live airbag module fromthe other automotive scrap pieces; and a sorting device for sorting thelive airbag module from the moving stream of automotive scrap pieces.12. The system as recited in claim 11, wherein the sorting includesdiverting the live airbag module into a receptacle along with automotivescrap pieces that are within a vicinity of the live airbag module. 13.The system as recited in claim 11, wherein the sorting is performedwithout activating the live airbag module.
 14. The system as recited inclaim 11, wherein the sorting device includes a sorting mechanism thatdiverts the live airbag module using a diverting force configured to notactivate the live airbag module.
 15. The system as recited in claim 14,wherein the sorting mechanism is a paint brush type plunger.
 16. Thesystem has recited in claim 12, wherein the live airbag module ispartially occluded by at least one other automotive scrap piece so thatthe vision system is unable to acquire spectral image data of anentirety of the live airbag module.
 17. The system as recited in claim16, wherein the artificial intelligence system is configured to identifythe partially occluded live airbag module and distinguish the partiallyoccluded live airbag module from the other automotive scrap pieces. 18.The system as recited in claim 17, wherein the artificial intelligencesystem is configured with a semantic segmentation algorithm fordistinguishing between live airbag modules and other automotive scrappieces.
 19. The system as recited in claim 11, wherein the artificialintelligence system is configured to classify a particular automotivescrap piece as a live airbag module if the particular automotive scrappiece sufficiently resembles a live airbag module.
 20. The system asrecited in claim 11, wherein the artificial intelligence system isconfigured to classify a particular automotive scrap piece as a liveairbag module in a ratio of false positives to false negatives greaterthan a predetermined threshold.